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ABSTRACT 


Since  the  utility  of  technical  manuals  is  greatly  influenced  by 
their  reading  difficulty,  a  survey  of  factors  affecting  readability  of 
technical  publications  was  conducted.  The  work  done  by  George  Klare 
was  relied  on  to  a  large  extent  for  a  bibliography  of  methods  of  measure- 
in,:  renda  tility  prior  to  1966.  The  survey  covers  additional  research 
done  since  that  time  with  particular  applications  toward  reading  technical 
materials.  The  report  concludes  with  suggested  areas  for  Human  Factors 
research  in  readability  of  technical  publications. 


Survey  of  Research  on 
Readability  of  Technical  Publications 


Introduction 


The  literature  on  readability  research  is  vast  but  only  a  very  small 
portion  has  any  direct  relation  to  technical  publications.  The  research 
on  readability  has  for  the  most  part  been  designed  around  isolating  factors 
affecting  reading  efficiency,  judgements  of  difficulty,  levels  of  compre¬ 
hension,  learning  and  retention.  The  scope  of  this  report  is  limited  to 
major  studies  in  the  readability  literature  which  appears  to  bear  most 
directly  on  problems  associated  with  the  use  of  technical  publications 
in  personnel  subsystems. 

In  personnel  subsystems  a  technical  publication  is  often  defined  as 
any  device  for  storing  information.  It  can  be  a  paper  manual,  a  micro¬ 
form  reader-printer,  a  visual-audio  instructional  system  or  a  computer 
storage  bank  with  a  printer  or  display  readout.  The  technical  publication 
is  used  for  one  or  more  of  three  major  purposes:  for  training,  as  a 
reference  in  supplementing  the  human  memory,  or  for  cooperative  aiding 
(e.g.  checklists,  step-by-step  procedures). 

There  are  three  general  categories  into  which  most  readability 
studies  fall.  First  of  all  there  is  a  sizable  group  of  studies  which 
primarily  indicate  the  legibility  of  either  hand  writing  or  typography. 
These  are  almost  entirely  perceptual  in  nature.  Human  Factors 
studies  on  readability  almost  always  refer  to  the  legibility  of  alpha- 
numerics  on  visual  displays.  Another  area  of  research  on  readability 
is  ease  of  reading  due  either  to  interest  value  or  pleasantness  of 
writing.  Only  one  study  was  found  in  this  area  with  technical  material 
as  content.  (Klare,  Mabry,  Gustafson,  1955).  The  remaining  category 
of  literature  on  readability  covers  work  done  to  indicate  ease  of  under¬ 
standing  due  to  the  style  of  writing.  Findings  from  this  last  category 
are  of  most  interest  from  a  technical  publication  view  point. 

This  report  covers  therefore,  those  classical  readability  factors 
which  apply  to  reading  material  having  technical  content.  In  addition 
it  provides  a  summary  of  major  techniques  which  can  be  used  in  the 
evaluation  of  technical  publication  readability.  The  report  concludes 
with  a  discussion  of  Human  Factors  research  needed  in  readability  of 
technical  publications. 

Classical  Readability  Factors 

According  to  Klare  (1963)  the  first  measures  of  readability  recorded 
were  that  of  the  Talmudists  in  900  A.D.  They  made  use  of  word  and  idea 


counts  so  that  the  frequency  of  occurance  could  be  used  to  distinguish 
usual  from  unusual  meanings.  In  the  U.  S.,  educators  have  been  interested 
in  readability  for  sometime  -  particularly  with  regard  to  children's 
readers.  Word  counts  were  popular  as  early  as  181*0  in  determining  reading 
levels  of  the  McGuffey  readers. 

Klare  cites  1921  as  a  milestone  in  readability.  During  that  year 
Thorndike  published  "The  Teachers  Wordbook"  wherein  he  tabulated  the 
frequency  with  which  words  occur  in  print.  This  probably  more  than  any 
other  one  thing  influenced  the  teaching  of  vocabulary  in  schools  and 
provided  the  base  for  readability  formulas  development. 

The  usual  factors  in  reading  material  which  have  been  idenified  as 
important  criteria  for  reading  difficulty  are  word  factors  and  sentence 
factors.  Apparently  word  and  sentence  factors  explain  about  as  much  of 
the  variance  in  performance  on  reading  efficiency  as  can  be  explained 
with  additional  style  factors.  According  to  Klare  (1963)  the  most  accurate 
formula  in  predicting  grade  levels  is  the  Dale  -  Chall  formula  which  uses 
only  two  factors  -  word  difficulty  and  sentence  length.  In  designing  an 
automated  readability  index  Smith  and  Sen ter  (1967)  provided  a  way  of 
mechanically  tabulating  these  two  factors  for  passages  as  they  are  typed 
on  a  standard  typewriter. 

Word  frequency  correlates  most  highly  with  recognition  time. 

Soloman  and  Postman  (1952,  in  Klare,  1963)  using  artificial  words 
(Turkish)  not  previously  seen  by  the  subject,  built  up  word  frequency 
usage  by  controlling  the  number  of  tachistoscopic  exposures  to  the  words. 
They  found  that  recognition  thresholds  vary  inversely  with  frequency  of 
prior  usage.  It  was  not  determined,  however,  to  what  extent  frequency 
of  prior  exposure  and  frequency  of  prior  response  interacted  in  the 
relationship. 

The  older  or  more  frequently  words  are  used,  the  shorter  they  tend 
to  become.  Television  has  become  TV,  radio  detecting  and  ranging  is 
radar  and  car  is  all  that  is  left  of  the  horseless  carriage.  Also  it 
is  generally  found  that  the  shorter  the  word,  the  shorter  the  recognition 
time.  More  often  than  not,  therefore,  shorter  words  are  judged  easier 
than  longer  words.  (Klare,  1963). 

Word  frequency  and  familiarity  of  words  to  a  reader  have  a  close 
relationship,  but  it  is  not  a  perfectly  linear  one.  Klare  (1963)  notes 
that  there  is  a  law  of  diminishing  returns  at  work  here.  Beyond  some 
point  of  high  frequency  one  additional  exposure  will  have  little  effect 
upon  recognition,  whereas  at  low  frequencies  one  additional  exposure 
can  be  highly  effective  in  producing  faster  recognition  times. 
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The  length  of  the  sentence  is  the  next  most  important  factor  in  the 
style  of  writing  for  reading  efficiency.  This  is  partly  due  to  structure 
of  sentences,  (complicated  sentences  are  gene  rally  longer  than  simple 
sentences)  but  not  entirely.  The  length  of  the  sentence  itself  because 
of  perceptual  memory  span  makes  sentence  length  a  major  factor  in 
readability  formulas  regardless  of  the  sentence  structure.  Miller  and 
Selfridge,  (1950,  in  Klare,  1963)  showed  that  memory  span  is  closely 
related  to  the  extent  to  which  organization  of  verbal  context  approximates 
English.  The  percent  of  recall  increased  as  the  order  of  approximation 
increased  (Nonsense  to  English).  Recall  decreased,  however,  as  sentence 
length  increased. 

A  sentence  factor  which  has  been  studied  in  several  different  ways 
is  redundancy .  Redundancy  is  the  extent  to  which  a  given  unit  of 
language  is  determined  by  nearby  units.  (KLare,  1963).  It  has  been 
approached  experimentally  by  the  degree  to  which  the  words  approximate 
English  (Miller  &  Selfridge,  1950),  by  deleting  letters  from  words 
(Chapanis,  195U)>  or  by  deleting  words  (Taylor,  1953;  Bormuth,  1967; 
Federman,  MacPherson,  and  Siegel,  1970).  The  latter  method;  known  as  the 
"Cloze"  technique,  has  been  shown  to  be  an  effective  method  of  evaluating 
surface  and  submarine  personnel  comprehension  of  Sonar  manuals.  When 
letters  are  deleted  from  words,  passages  that  are  easy  to  reconstruct 
are  not  necessarily  those  easiest  to  read.  Chapanis  (1951;)  gave  13 
English  prose  passages  to  subjects  using  both  random  and  regular  patterns. 
He  found  that  in  general  people  do  poorly  in  recons time ting  passages  when 
the  amount  of  material  deleted  is  30$  or  more. 

Special  consideration  of  the  reader  is  required  in  assessing  the 
readability  of  written  material.  Reading  level  roughly  corresponds 
to  the  last  school  grade  completed.  This  will  vary  depending  on  any 
special  experience  the  reader  has  with  the  material  and  with  general 
vocabulary  and  IQ  ratings.  (Klare,  1963).  Both  high -ability  and  low- 
ability  readers  can  benefit  from  more  readable  material,  however.  No 
matter  what  a  reader's  skill  level,  if  the  same  information  content  is 
presented  in  a  more  readable  manner,  he  will  be  able  to  read  it  with 
greater  ease  (Klare,  1963). 

The  difference  between  easy  and  hard  versions  of  material  in  terms 
of  test  scores,  may  not  be  reflected  in  comprehension  tests.  This  is 
particularly  true  with  technical  material.  A  technician's  special 
knowledge  of  electronics  may  allow  him  to  get  the  same  comprehension 
score  on  electronics  passages  regardless  of  style  difficulty.  (Klare, 
Mabry,  Gustafson,  1955).  When  an  eye-movement  camera  is  used,  however, 
easy  style  technical  material  produced  higher  scores  than  the  hard  style 
on  words  per  fixation  (Klare,  Shuford,  Nichols,  1957). 
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Motivation  is  a  major  factor  in  comprehension,  learning  or  retaining 
information.  Subjects  with  either  strong  or  weak  sets  to  learn  material 
wil_  nvid  faster  and  with  fewer  fixations  only  on  easy  versions.  Reading 
speed  may  decrease  on  hard  versions,  with  increased  learning  motivation. 
(Xlare,  19  0).  But  Klare  (1963)  indicates  that  recall  scores  will  be 
relatively  hither  on  both  hard  and  easy  versions  where  there  is  motivation 
to  learn. 

A  principle  of  least  effort  is  also  at  work  with  most  readers.  All 
other  things  >eing  equal,  a  reader  under  low  motivation  will  choose 
material  around  two  grade  levels  below  his  level  when  reading  for  pleasure 
(Klare,  19b3).  In  other  words,  technical  readers  are  not  likely  to  read 
difficult  technical  information  (e.g.  circuit  theory)  unless  the  topic  is 
very  interesting,  very  necessary  to  something  they  value  highly  (e.g.  health, 
life,  family)  or  they  are  strongly  motivated  to  learn. 

Most  of  the  readability  research  has  been  primarily  on  the  difficulty 
of  style  of  writing.  Klare  (1963)  discusses  other  important  factors  present 
in  determining  the  readability  of  a  publication.  Broadly  these  are  content, 
format,  and  organization  of  information  presented. 

In  summary,  the  following  general  rules  can  be  applied  to  technical 

publications . 

1.  In  considering  reading  speed  and  efficiency,  more  readable  material 
provides  consistantly  for  an  increase.  This  holds  true  for  most  educational 
levels  and  amounts  of  background  readers  have.  This  is  particularly  im¬ 
portant  in  motivating  voluntary  reading  of  difficult  material  such  as  the 
theory  of  operation  found  in  maintenance  manuals. 

2 .  Readers  as  a  group  tend  to  rank  materials  in  terms  of  readability 
ease  in  much  the  same  order  as  a  writer  using  readability  formulas.  Read¬ 
ability  formulas  can  give  gross  indications  of  acceptability  of  material. 

3.  If  a  publication  is  to  attract  a  wide  readership  (e.g.  operators  - 
maintainers  of  equipment)  there  will  be  low  education  levels  involved. 

There  will  be  generally  a  low  set  to  learn  and  a  large  amount  of  the  principle 

of  least  effort. 

a.  Roadability  formulas  will  not  generally  predict  how  well  material 
will  be  conprenended.  No  matter  what  the  style,  the  material  may  be 
comprehended  the  same  because  a  reader  may  be  highly  motivated  to  learn 
maie rial,  he  may  have  a  large  amount  of  experience  or  background  with  the 
topic  being  covered,  or  he  may  read  a  passage  over  and  over  until  he  under¬ 
stands  what  is  meant.  For  low  motivate!,  inexperienced,  and  time  limited 
individuals,  however,  the  more  readable  the  style,  the  better  the  compre¬ 
hension,  rate  of  learning,  and  level  of  retention. 


Keadability/Comprehensibility  Measurement  Techniques 


1.  Prediction  formulas  -  The  evaluation  of  readability  or  compre¬ 
hensibility  of  textual  material  may  be  determined  with  one  or  more  of  the 
following  methods:  expert  judgement,  simulated  field  conditions,  pre¬ 
diction  formulas  and  the  "cloze”  technique. 

In  the  past  prediction  formulas  have  been  used  most  widely  in  assessing 
educational  materials.  Expert  judgements  or  simulated  field  conditions 
are  usually  employed  however,  in  verifying  technical  publications .  This 
is  partly  because  readability  formulas  appear  to  be  weak  as  indicators 
of  readability  of  technical  material  (Ross,  1959)  and  non-fiction  special¬ 
ized  material  like  physics  text  books  (Marshall,  1957). 

Probably  the  most  popular  formula  is  that  of  Rudolf  Flesch  (19U8). 

It  has  attained  great  popularity  because  it  is  a  simple  formula  consisting 
of  only  3  factors;  it  correlates  highly  ( . 7li)  with  McCall -Crabbs  Standard 
Test  Lessons;  and  Flesch  popularized  it  with  a  series  of  articles  and 
books  getting  support  of  writers  in  journalism,  government  and  business. 

The  general  procedure  is: 

(1)  Select  samples  of  100  words  throughout  material. 

(2)  Compute  average  sentence  length  in  words  (Xs) 

(3)  Count  affixes  Xm 

(li)  Count  personal  reference  (Xh) 

The  original  formula  (19U3)  is: 

X  =  .0?  Xm  +  .07  Xs  -  .05  Xh  +  3.27 

There  is  a  shortened  fonnula  for  reading  ease  RE  =  206.835  -  .82j6  wl  -  1.015'  si 
Where : 

wl  -  number  of  syllables  per  100  words 
si  -  avg.  number  of  words  per  sentence 

The  shortened  fonnula  probably  is  the  most  applicable  to  technical  publi¬ 
cations  because  it  eliminates  human  interest  factor.  In  technical  writing 
human  interest  (i.e.  you  turn  the  crank)  versions  have  been  judged  less 
acceptable  >y  airmen/  [klare,  Mabry,  Gustafson,  May  1955) 

The  next  most  popular  formula  is  the  Dale-Chall  (I9h8). 
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The  Dale-Chall  formula  is: 


Xc  s  -1579  X1  +  .OU96X2  +  3*6365 
Where : 

%2  ~  average  sentence  length  in  words 

Xc  =  Reading  grade  score  of  pupil  who  could  answer  one -half  test  questions 
50  on  McCall  -  Crabb  Test 

X-^  =  %  of  words  outside  Dale  list  of  3000 

A  formula  which  has  received  acceptance  in  Air  Force  Standard  Writing 
guides  is  the  Fog  index  discussed  in  Grunning  (1952).  The  procedure  is 
similar  but  simpler  than  the  Flesch  method. 

Fog  Index 

(1)  Take  100  word  samples 

(2)  Divide  the  number  of  words  by  number  of  sentences  to  get  average 
sentence  length 

(3)  Count  number  of  words  by  three  or  more  syllables 
(1*)  Total  2  factors  and  multiply  by  .1| 

A  final  formula  is  the  Gray  &  Leary 

X,  =  -.01029X2  +  .009012X^  -  .0209UX6 
-.03313X7  -  .OlU85X85+  3* 7 7U 

The  work  of  Gray  &  Leary  (1935)  in  Klare,  1963  is  considered  a  landmark  on 
readability  because  of  the  detailed  analysis  they  used. 

The  above  formula  is  a  regression  equation  developed  from  a  ijli  Factor 
analysis  where : 

X^  =  Average  Comprehension  score  (Fiction  -  Non-fiction) 
for  adults  of  limited  reading  ability 
JO,  =  #  of  difficult  hard  words  not  on  the  Dale  list  of  769  words 
X^  =  #  of  personal  pronouns 
X^  =  Average  #  of  words  per  sentence 
Xy  =  %  of  different  words 
Xg  =  #  of  prepositional  phrases 

2.  Cloze  Techniques 

A  method  which  is  considered  a  possible  break  through  in  techniques 


1.  . 


for  evaluating  readability  of  technical  publication  is  the  "cloze" 
technique.  The  procedure  was  developed  by  Taylor  (1953)  but  has  only 
recently  been  applied  to  the  evaluation  of  technical  manuals  (Federman, 
MacPherson  and  Siegel,  1970).  The  technique  simply  requires  deletion  of 
words  from  a  passage  and  then  scoring  the  reader  on  the  percentage  of  words 
he  can  correctly  replace.  According  to  Taylor  (1957,  in  Federman, 

MacPherson  and  Siegel,  1970)  the  cloze  procedure  assumes  that  there  is 
a  high  correlation  between  how  readable  a  piece  of  writing  is  and  how  well 
it  can  be  understood  with  words  left  out.  The  more  a  person  understands  a 
piece  of  writing  the  more  likely  he  can  guess  the  missing  words. 

The  cloze  procedure  correlates  highly  with  test  comprehension  scores. 
Bormuth  (1967)  for  example,  reports  a  product  moment  correlation  of  .9k 6 
for  Uth  and  5th  grade  comprehension  on  multiple  choice  test  with  cloze 
scores.  Bormuth* s  technique  with  cloze  scores  involves  selectively 
removing  words  from  the  sample  to  reduce  the  effect  from  redundancy 
rather  than  understandability  suggesting  the  missing  words.  (Klare,  1966). 

3.  Use  Tests,  Expert  Judgements  and  Comprehension  Tests 

Actual  vise,  expert  judgement,  and  comprehension  tests  have  been  utilized 
extensively  in  evaluating  technical  manuals  (federman,  MacPherson  and  Siegel). 
Ross  (l 959)  developed  a  weighted  check  list  for  evaluating  technical  manuals 
covering  readability  of  technical  manuals.  The  check  list  depends  upon  a 
combinatiai  of  expert  judgements,  the  Fog  Index,  use  of  peak  stress  (under¬ 
lining  important  words  in  sentences)  and  use  of  personal  pronouns  in  the 
publication.  The  checklist  also  includes  factors  for  layout,  organization, 
decision  making  procedures  and  pictoral  or  schematic  illustrations  in  de¬ 
termining  the  comprehensibility  of  technical  material.  Many  factors  other 
than  written  language  enter  into  the  determination  of  the  comprehensibility 
of  technical  manuals .  These  are  covered  elsewhere .  (Folley  and  Munger,  1961 ) 

There  are  problems  associated  with  most  expert  judgements,  actual  use 
tests  in  simulated  field  conditions,  and  comprehension  tests  that  make  them 
undesirable  as  evaluation  techniques.  Simulating  the  actual  use  of  a 
technical  manual  is  costly,  inappropriate  early  in  system  development,  and 
confounds  technician  ability  with  manual  inadequacy.  Comprehension  tests 
are  unstandardized  from  manual  to  manual  and  may  not  reflect  readability 
faults.  Expert  judgements  are  undesirable  because  of  problems  in  selecting 
experts,  defining  valid  criteria  measures,  and  in  generalizing  from  expert 
opinion  to  the  user  population.  (Federman,  MacPherson  and  Siegel,  1970). 

Areas  for  Research  in  Readability  of  Technical  Publications 

Far  too  little  research  has  been  done  with  readability  of  technical 
publications.  As  Klare  (1963)  points  out,  most  readability  research  is 
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•entered  around  difficulty  of  style  with  almost  no  research  on  measures  of 
conwr.t.  Unless  content  is  known  not  to  change  when  style  is  changed,  it 
ic  difficult  to  attribute  changes  in  comprehension  scores  to  changes  in 


Chapanis  (1965)  feels  that  all  of  the  present  readability  formulas 
including  the  cloze  techniques  are  virtually  useless  to  our  problems  in 
most  instructional  material.  He  states  that  we  should  be  interested  in 
the  intelligibility  or  understandability  of  what  is  said  rather  than 
reading  speed,  comprehension  of  text  material  or  judgement  of  difficulty. 
The  close  technique  although  useful  in  evaluating  technical  manual  sections 
appears  to  be  of  little  value  in  short  instructions .  The  cloze  procedure 
as  presently  designed  requires  large  samples  (usually  250  words)  whereas 
our  samples  of  language  in  instructional  material  are  often  short. 

Ir.  experiments  which  have  been  designed  to  find  out  the  best  ways  of 
presenting  Instructions  to  bring  about  the  specific  human  actions  intended, 
the  results  have  been  gratifying.  Only  two  such  experiments  have  been 
found,  however.  Conrad  (1962)  found  that  a  significantly  greater  number  of 
phone  calls  could  be  successfully  transmitted  with  simplified,  clear 
instructions  than  could  be  made  with  mechanical  changes  in  typography. 

Haney  (1969)  found  that  when  information  is  action  sequenced  testing 
performance  is  increased.  A  tabular  format  with  well  defined  step  by  step 
instructions  was  superior  to  standard  format.  Haney  emphasized  that  the 
information  had  to  be  action  sequenced.  It  is  not  enough  to  draw  lines 
down  a  page  or  put  sentences  in  a  column. 

Recent  research  in  psycholinguistics  such  as  syntactic  language 
structuring  around  a  "kernel"  of  language  provides  a  method  for  developing 
rules  for  structuring  instructions  (Miller,  1962,  1969).  Miller  (1962) 
has  shown  that  single  active  fom  sentences  are  interpreted  faster  than 
negative  or  passive  forms.  Chapanis  (1965)  encourages  experimental  work 
be  done  with  complicated  sentences  and  then  be  tested  out  in  terms  of  the 
effects  on  the  behavior  of  ordinary  people.  Perhaps  the  degree  of  compli¬ 
cation  of  a  sentence  can  be  quantified  in  terms  of  some  unit  similar  to 
information  bits.  As  sentences  have  more  than  one  adjective  modifying 
a  noun  or  more  than  one  object  of  a  verb,  the  number  of  alternatives 
increase,  therefore  decreasing  the  rate  at  which  the  information  can  be 
processed. 

Chapanis  (1965)  suggested  several  areas  for  human  factors  research 
which  still  need  to  be  carried  out.  As  a  measure  of  intelligibility 
the  cloze  technique  should  be  validated  on  various  technical  manuals  with 
different  manual  users.  Comparison  of  the  Taylor  and  Bormuth  forms  of 
cloze  or.  technical  publications  may  be  valuable  in  obtaining  a  more  valid 
evaluation  ~ool.  Short  instructions  built  up  on  the  basis  of  task  analysis 
similar  to  that  of  Haney  or  Conrad  should  be  investigated  further. 
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Criteria  for  evaluating  manuals  in  terms  of  the  content  and  under- 
standability  of  the  material  is  needed.  Methods  of  testing  manuals  and 
guides  for  preparing  them  are  in  great  demand. 

Other  research  is  needed  to  decide  what  mixture  of  pictures,  schematics 
and  words  provide  the  most  understandable  combination. 

Special  word  lists  are  needed  for  special  purposes.  Words  that  are 
likely  to  be  within  the  comprehension  of  the  average  maintenance  technician 
should  be  compiled. 

To  date  there  is  almost  nothing  on  the  readability  of  foreign  languages. 
As  we  start  designing  machines  and  instruction  manuals  for  use  by  diverse 
nationalities,  research  problems  become  evident.  Perhaps  instructions 
consisting  entirely  of  pictures  would  be  appropriate  for  guiding  specific 
human  actions  to  be  taken  in  their  proper  order. 

Finally  research  on  what  motivates  people  to  use  a  technical  publi¬ 
cation  is  badly  needed.  How  can  publications  be  designed  to  instill 
interest,  or  provide  a  strong  set  to  learn?  Are  colors  and  special 
bindings  motivating?  Will  a  maintenance  man  prefer  comic  characteristics 
to  plain  pictures?  Is  a  sense  of  accomplishment  with  a  technical  publi¬ 
cation  rewarding?  Will  a  maintenance  man  operate  buttons  on  a  machine  to 
find  information  more  often  than  pick  up  an  index  to  a  paper  publication? 
Can  special  training  techniques  be  designed  to  encourage  motivation  to 
learn  how  to  use  technical  publications  properly? 

Systematic  research  into  any  of  these  problems  should  reveal  basic 
behavioral  relationships  in  a  very  important  but  neglected  area  of  man- 
machine  communications  -  an  area  which  in  the  past  has  relied  almost 
exclusively  on  the  individual  judgments  of  technical  writers  and  graphic 
artists . 
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